Worried evaluator:

The evaluator's job when analysing evaluation material is to get evaluation insights: to code the data, weigh the evidence, build the argument, communicate the findings. If an AI model can do all that, what is left for us?

This question starts from the wrong place.

The evaluator in the attic#

Think of how a lot of evaluation interview and document analysis has worked: An evaluator holes up for forty days, codes the interviews and documents, and ends up understanding the programme better than anyone. Then comes the hard part: getting that understanding across to the people who need it. Usually it goes into a report they never read, perhaps with a stakeholder workshop at the end.

Of course, the understanding ends up in the wrong head. The one person who was never part of the programme gets all the insight, and the people who are part of it get a summary.

What evaluation was always for#

Evaluation was always in fact about facilitating changes of mind within, and of relationships between, the people who matter, not in the evaluator. Those people are the programme's staff, partners, participants and funders: what they know and believe, what they do, how they work together, their shared models of how things work and their visions of how things should work. Those understandings and relationships are, in a way, what a programme is.

We do not need to "include" these people. They are already there, doing the work, as well as giving the interviews, writing the reports, taking part. Most evaluators know this and try to start from where people are, facilitating learning.

What AI changes#

If a model can take on much of the old-school forty days in the attic, in a form that others can check, the evaluator's time can go where it should have gone all along: helping those people work through the evidence together, in the analysis, the interpretation and even the coding, not only in a feedback session on an almost finished report. AI can help us not to grind people down with overwhelming amounts of data, but prestructure what we ask them to do where their experience can add the most value and conversely, where their participation in the process is most likely to benefit them in terms of improving their (individual and collective) understanding and learning.

So in the best case, AI lets us do our real job better. People have argued for evaluation like this for decades; what is new is that maybe AI can support us in doing a better job of involving people in in-depth analysis and sensemaking by redesigning evaluation procedures, not just adding AI to the old ones.

I first set this out after EES 2026, in a LinkedIn article.

I sometimes use AI to help draft these pages, but the final text is ours.